ShowTable of Contents
Introduction
Lotus Connections is based on IBM WebSphere® Application Server (WAS). It has 11 features, each of which can have its own cluster. Because Connections 3.0 supports only Network Deployment, the smallest topology would be a single node cluster and features on the same server instance.
No pilot or standalone deployment is provided because users typically choose a clustered deployment for their production environment, like that shown in figure 1. The figure depicts a two-node topology, but actually one Deployment Manager (DM) can manage many nodes.
Figure 1. Typical cluster deployment
Traditionally, in pre-3.0 Connections versions, you ran the installer on each node to install it (see figure 2). So, if you had five nodes that must be clustered, you needed to run the installer on each node, for a total of five times. Assuming that one installation takes 1 hour, it would take 5 hours---not counting the debug time if one installation fails. Obviously, this long wait time is not desirable.
Figure 2. Cluster deployment process before Lotus Connections 3.0
Moreover, the “old” process for cluster deployment is basically to install a standalone first, and then federate the node into the DM; however, many issues have been reported when doing the federation. Fortunately, things changed in Lotus Connections 3.0.
WAS synchronization mechanism
First we need to discuss the WAS synchronization mechanism because this is the technique we use for fast cluster deployment. Specifically, we must introduce the two concepts of epochs and digests.
Epochs
At a high level, you can think of epochs as a cache mechanism used by the node agent and DM to determine what files in the master repository have changed since the last synchronization operation.
When the next synchronization operation is invoked, only folders in the cache will be compared. If the epochs for a folder in the cache are different, such folders are considered as having been modified and will be checked further later on in the synchronization process.
At a more detailed level, epochs (implemented by com.ibm.websphere.management.repository.ConfigEpoch) are objects containing a long type of variable that, when first initialized, has a value of System.currentTimeMillis. Updating an epoch will increase the long variable by 1, whereas refreshing an epoch will reset the long variable to System.currentTimeMillis.
There are two different types of epochs, repository epoch and folder epoch. A repository has an epoch associated with it, referred to as a repository-level epoch; each folder in the repository has an epoch associated with it, referred to as a folder-level epoch.
There are two sets of cell-level and folder-level epochs. One set is on the DM repository; another set is on the node repository. When you compare epochs, you are comparing the epoch from one repository with that of the other repository.
For example, you would compare the cell-level epoch of the master repository with the cell-level epoch of node repository, and you would compare a folder-level epoch of the master repository with the same folder-level epoch of the node repository.
Digests
Checking digests (checksums) is how synchronization decides whether two files are the same. If the digests of two files are different, synchronization considers files to be different and will update the node repository. Otherwise, it thinks two files are the same, and no file transfer will occur.
The actual digest calculation is implemented through Java APIs (see com.ibm.ws.management.repository.DocumentDigestImpl, calc() methods and getMessageDigest() method). Listing 1 shows how synchronization calculates the digest.
Listing 1. Code to calculate the digest
MessageDigest oneMessageDigest = MessageDigest.getInstance("SHA"); <-- digest algorithm is "SHA"
if (oneMessageDigest != null)
{
// Get the digest for the data in the stream
while ((bytesRead = input.read(buffer)) > 0)
oneMessageDigest.update(buffer, 0, bytesRead);
digest = oneMessageDigest.digest(); <-- calculate digest
}
}
Synchronization
Synchronization operations are invoked by the node agent, in the case of autosync or of user explicitly initiated synchronization, etc; or by the syncNode/addNode processes when syncNode.bat/sh or addNode.bat/sh is invoked. In all synchronization scenarios, the process initializing the synchronization operation communicates with the DM.
The DM retrieves information about the state of the master repository and compares it with the node repository. A list of changed folders is returned to the node agent after the epochs of the two repositories are compared.
Next, for each folder in the list, DM compares the digests of the documents to determine whether the files are indeed different. The changed files are transferred to the node via file transfer and checked into the node repository by the node agent.
A synchronization operation can be started as long as there is not another synchronization operation already going on; if there is, you must wait for the on-going synchronization operation to finish before a new one is started.
Fast cluster deployment
Let's now discuss the steps we use.
Process improvement
All configurations with WAS will result in changes in the config files. Because a cluster will share the same set of WAS configurations, the synchronization mechanism will synchronize all those configuration files from the DM to each node.
So if we install on the DM, then every configuration will be synchronized to each node, and there is no need to reinstall on each node. In this way, for a product like Lotus Connections, the installation steps are as follows:
- Install WAS and add all the nodes into the DM.
- Configure LDAP to the DM.
- Run Lotus Connections installer on the DM machine, to deploy all applications onto the DM.
- After installation, start the node agents, which will invoke the synchronization automatically.
- Roughly half an hour later (depending on the Web performance), the synchronization finishes, and you can start the cluster and log in to Lotus Connections.
So if we again assume the installation takes 1 hour, then for a cluster deployment with five nodes, this fast cluster deployment will need only 1.5 hour. We estimated the previous method taking 5 hours, so this is a significant improvement in installer performance (see figure 3).
Figure 3. Improved cluster deployment process in Lotus Connections 3.0
By providing this fast cluster-deployment method, Lotus Connections can offer a more centralized deployment, which means everything hooks up with the DM. Users can complete the installation and manage nodes and applications from the DM, and future upgrades/migrations will also be run on the DM.
Resource differences
To achieve this new install method, resources are created either on the cell level or cluster level, to ensure the visibility across all nodes. This is a big difference from the old deployment. No resource is created for the node level and, unless required, no resource is created for the server level.
Table 1 lists all these resources in Lotus Connections 3.0.
Table 1. Connections 3.0 resources
Cell-level resources | Cluster-level resources |
Security Settings | Async Beans |
Service Integration Bus | Schedulers |
Java Mail |
|
WAS Variable |
|
JDBC Resources |
|
JMS Resources |
|
Limitations
The synchronization can synchronize the configuration files under AppServer\profiles\<profile name>\config. If you have configuration files that are not in this directory, you may need to copy them manually on each node. Or, for example, if you have lib files in a separate directory under AppServer, they also will not be synchronized onto each node.
If you start the cluster before the synchronization finishes, it will cause problems; specifically, the synchronization will be interrupted, and you would probably find that not all applications have been synchronized to nodes. You can check AppServer\profiles\<profile name>\config\cell\<cell name>\applications to verify all applications are there.
Conclusion
Using the WAS synchronization mechanism, Lotus Connections 3.0 provides a fast means of cluster deployment that is easy for users to implement and scale, greatly reducing the time required for install.
Resources
developerWorks IBM Connections product page:
https://www.ibm.com/developerworks/lotus/products/connections/
Connections product documentation:
http://www-10.lotus.com/ldd/lcwiki.nsf/xpViewCategories.xsp?lookupName=Product%20Documentation
Connections Forum:
http://www-10.lotus.com/ldd/lcforum.nsf?OpenDatabase
About the authors
Hao Zhang is a former Software Engineer on the IBM Connections Install team who now is a Development Manager. He served on the Install team since the first release of Lotus Connections, leading the team in continuously improving the usability and consumability of the Connections installer. You can reach Hao at
haozhang@cn.ibm.com.
Jing Yao is a Software Engineer working with the IBM Connections Install and Lotus Quickr Install team in Shanghai, China. She has three years of experience in install technology and has extensive knowledge of WebSphere Application Server deployment and configuration. You can reach Jing at
yaojing@cn.ibm.com.